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Abstract. Many complex systems obey to optimality conditions that 
are usually not simple. Conflicting traits often interact making a Multi 
Objective Optimization (MOO) approach necessary. Recent MOO re¬ 
search on complex systems report about the Pareto front (optimal de¬ 
signs implementing the best trade-off) in a qualitative manner. Mean¬ 
while, research on traditional Simple Objective Optimization (SOO) of¬ 
ten finds phase transitions and critical points. We summarize a robust 
framework that accounts for phase transitions located through SOO tech¬ 
niques and indicates what MOO features resolutely lead to phase transi¬ 
tions. These appear determined by the shape of the Pareto front, which 
at the same time is deeply related to the thermodynamic Gibbs sur¬ 
face. Indeed, thermodynamics can be written as an MOO from where 
its phase transitions can be parsimoniously derived; suggesting that the 
similarities between transitions in MOO-SOO and Statistical Mechanics 
go beyond mere coincidence. 

Keywords: multiobjective optimization, Pareto optimality, phase tran¬ 
sitions, statistical mechanics, thermodynamics 


1 Introduction 

Optimization has always been a major topic in complex systems research. Op¬ 
timality conditions are relevant for a wealth of biological [n, m, la, i, m and 
other natural and synthetic systems 0, H, i, 0, unj. Evolution through nat¬ 
ural selection is a main driver of biological systems towards optimal designs m 
[12j and certain physical principles (e.g. maximum entropy or optimal diffusion 
structures) already introduce a bias towards functional extrema. Human-made 
systems are equally constrained through cost-efficiency calculations - e.g. in 
transportation networks 0, m- 

Describing these situations requires optimal designs that often cope with 
interacting constraints. To give a good account of these selective forces, a Pareto 
or Multi Objective Optimization (MOO) approach can be useful. Let us introduce 
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this theory through a recent relevant example | 13 j . (Technical definitions follow 
below.) Consider the set (F) of all connected networks with a fixed number of 
nodes (7 € F, figure [^). Among them we seek those minimizing the average 
path length ( 1 ) (7) and the number of edges p{'j). These are the target functions 
(r/(7) = {ti = {l){"f),t2 = p(7)}) of our MOO problem. A fully connected 
clique minimizes the average path length, but we need to implement all possibles 
links, which is costly. The minimum spanning tree has the least number of edges 
possible but its average path length is quite large. Take networks 71 and 72 
trading between these extremes and such that { 1 ) (71) < (/) (72) and p{ji) < 
p(7i). This means that 71 implements a better tradeoff than 72 and we say then 
that 7i dominates 72 (figure [^). A network € 7 T) C T not dominated by 
any other 7 G T is Pareto optimal. Often we cannot choose between a pair of 
networks because one is better than the other with respect to a target and worst 
with respect to the other - i.e. they are mutually not dominated. Because this 
situation is common, MOO solutions are often not a single global optimizer, 
but the collection of Pareto optimal (mutually non- dominated) networks that 
implement the most optimal tradeoff possible. We name this Pareto optimal set 

n c F. 

Target functions map each network 'y G F into a point of {{ 1 ) {j),p{j)). 
This plane, with the relevant traits in its axes, constitutes a morphospace in 
which salient network topologies are located as a function of their morphology 
[H] (figure [^). Morphospace of other systems visualize phenotypes or designs 
with respect to relevant properties. The Pareto optimal set is mapped onto T/( 7 T) 
and constitutes a boundary of the morphospace (figure [^-c). also known as the 
Pareto front. 

Some authors are beginning to explore the consequences of Pareto optimal¬ 
ity in biological systems 0, 0, 0 or in relevant models such as networks | 15 j . 
[ 16 ] or regulatory circuits 0, m- While they tackle relevant questions through 
MOO methods, the description of these optimal designs is often a qualitative 
account of the elements along the Pareto front (as in the study of a restricted 
morphospace - an interesting contribution nevertheless). The same qualitative 
bias appears in classic MOO literature. Is a more quantitative analysis possible? 
Are there universal features that reach through different MOO problems, thus 
uniting Pareto optimal systems despite their differences? Through our research 
[T 5 ] (sketched in section we have found a connection between MOO and 
statistical mechanics. Those universal features we were looking for are phase 
transitions and critical points, which leave clear imprints in the shape of the 
Pareto front. Some authors had explored MOO with Single Objective Optimiza¬ 
tion (SOO) methods - e.g. by integrating all targets linearly to define a global 
energy function Q{A) = with A = {Ai,..., A^} arbitrary parameters 

that introduce a bias towards some of the targets. Such research often finds 
phase transitions and other phenomena 0, nnj. A parsimonious theory lacked 
as to why some systems would present such transitions and others would not. 

To the best of our knowledge, authors researching MOO do not exploit this 
connection with thermodynamics which, we believe, much enriches the discus- 
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sion of Pareto optimal designs. Two relevant examples from network theory: 
The efficiency of different topologies has been researched for the relay of in¬ 
formation across a network using two distinct delivery heuristics m- It was 
made an exhaustive work in describing network topologies and locating them 
in a morphospace in which different network features are segregated. In that 
same morphospace, the Pareto fronts in m strongly indicate the presence of 
first and second order phase transitions. If the information theoretical aspect 
of the diffusion of messages across the network is considered, those transitions 
might become thermodynamically relevant. Similarly, the tradeoff of topological 
robustness when random or targeted nodes are taken away results in a Pareto 
front m- Under the light of our findings, second order transitions are present 
show up in that study. Also a first order transition exists that vanishes as the 
average degree of the network changes, suggesting a critical point. We further 
illustrate our findings with other two examples in section [2.1| 

A theory about phase transitions must fit within thermodynamics. For us, 
this is achieved due to the equivalence between the Pareto front and the Gibbs 
surface [18], [T9|, [20], an object known to embody phase transitions in its con¬ 
cavities and non-analyticities. We discuss thermodynamics in section |3.1[ not 
because our theory modifies previous knowledge about it, of course, but because 
in showing that phase transitions arise in thermodynamics precisely in the same 
way as in MOO, we place our findings for MOO on very solid ground. 


2 Theoretical Framework 

In this section we expand the loose introduction of MOO above. More details 
and methods can be found in the exhaustive literature |2T|, |22|, |23|, |24|, [25] . 
We assume minimization unless indicated otherwise. 

Consider a set X of possible designs x G X. In the example above, X = F 
is made of network designs. This will be used again later, along with another 
example in which X = A stands for all possible languages a G A derived from 
a mathematical computational model of human communication |8]. Within X 
we seek those optimal designs G U) G X that simultaneously minimize a 
series of target functions {Tf = {ti,... Ak})- These tk G Tf map each design 
X G X into target space {Tf{x) = {ti(a:),... ,tKix)} G K^), a morphospace of 
the system under research. 

Pareto dominance is defined in this target space. Take x, z G X. x dominates 
z (noted a; ^ z) if tk{x) < ti^{z) for all k and < tk'{z) for at least one k'. 

This means that x is objectively better than z. If given two designs {x,y G X) 
none dominates the other {x yi. y yi. x)^ we cannot chose one of them without 
introducing a bias towards some of the target functions. Pareto optimality is 
solved by putting choices between mutually non-dominated designs on hold. 

The Pareto optimal set 11 G X is such that every element z G X, z ^ TI 
is dominated by some x G 11 while any x,y G FI are mutually non-dominated. 
The projection Tf{II) conforms a, {D < K — I)-dimensional surface in that 
embodies the most optimal tradeoff possible between the targets. Moving along 
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Fig. 1. Phase transitions in Pareto optimal systems, a In a design space of 
complex networks a set of weighted target functions defines a global energy (equation 
and renders a potential landscape (explored in [18], [l3]). b Those same targets map 
the design space into the target space. The set of Pareto optimal designs is mapped 
onto a boundary of this morphoscpace: the Pareto front, which represents the most 
optimal tradeoff between the targets, c The concept of dominance is geometrically 
simple in target space, d Energy minimization for fixed A returns a single point of the 
Pareto front. Changing A we visit different solutions. Depending on the shape of the 
Pareto front, second (e) and first (f) order phase transitions arise as a function of A. 
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the front it is impossible to improve all targets at once: an increment in at least 
one tk is necessary if we wish to decrease some other tk' ■ 

We sketch now the basic situations of our theoretical framework that connects 
the Pareto front and thermodynamics. We refer the reader to [TS] for a more 
exhaustive discussion. 

The simplest SOO problem that includes all MOO targets defines a linear 
global energy function: 


= ^Xktk{x), ( 1 ) 

k 

where A = {A^; fc = 1,..., it'} are parameters that bias the optimization towards 
some of the targets. We say that equation[^has collapsed the MOO into an SOO. 
A set A with fixed values \k defines one single SOO, thus equation(with free 
Afc) produces indeed a family of SOOs whose members are parameterized through 
A. We will study: those SOOs, the constraints that the Pareto front imposes to 
their solutions, and the relationships between different SOOs of the same family. 
The validity of the results holds for any positive, real set A. For convenience, 
though: i) We take K = 2, which simplifies the graphic representations and 
contains the most relevant cases, ii) We require J2k ^ without loss of 

generality. For K = 2 then Ai = A, A 2 = 1 — A, and = Xti + (1 — X)t 2 - iii) We 
impose Xk ^ 0 Vfc, thus A S (0,1). Comments about fringe cases can be found 
in [H]. 

As said above, for given Xk one fixed SOO problem is posed. Then, equation 
[^with fixed i? defines eguifitness surfaces noted Tyi(f?). Each ryi(f7) constitutes 
a (AT — l)-dimensional hyperplane in target space. For K = 2 these surfaces 
become straight lines (figure [^): 

rA(f2) = |(ti,t2)| ■ (2) 

The slope of T\{n) along each possible direction tk in the target space only 
depends on A (here, dt 2 /dti = —A/(l — A)). Different T\{n) for fixed A are 
parallel to each other. The crossing of tx{[2) with each axis is proportional 
to f} (from equation the crossings with the horizontal and vertical axes read: 
17/(1 —A) and 17/A). For a given SOO (constant A), minimizing 17 means finding 
ta(17) with 17 the lowest value possible such that ta(17) still intersects the Pareto 
front (figure [^) . This is equivalent to pushing the equifitness surfaces against 
the Pareto front as much as possible thus lowering the crossings with the axes. 

The SOO optimum xx G II lays at the point Tf{xx) at which ta(17) is 
usually tangent to the front (figure [^). The exceptions to this rule are the most 
interesting cases. The solutions to different SOOs (defined by different values of 
A) are found in different points along the front. For A € (0,1), equifitness surfaces 
present a slope — A/(l — X) = d G (— 00 ,0) {d decreases as A increases). Consider 
now differentially small modifications of A. This allows us to drift infinitesimally 
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slow through the SOO family. We could expect that solutions between different 
SOOs will change so gradually as well, but that is not always the case. 

The front in figure]^ is convex (with respect to the optimization direction 
determined by A S (0,1)). Its slope spans the whole range d S (—oo,0). This 
guarantees that, as we drift through A, each different SOO problem has one 
characteristic solution laying exactly where the equifitness surface is tangent to 
the front. We can sample the front smoothly, thus anything that we measure on 
the SOO solutions (i.e. any order parameter) will be a smooth function of A as 
well. Convex Pareto fronts whose slope span the whole range d S (—oo,0) do 
not present any accident. 

Consider now the case in figure [^. It represents a convex front whose slope 
spans d S {—oo,d* < 0). For A S (A* = —d*/(I — d*), 1), we can pose different 
SOOs whose solutions lay at different points of the convex part of the front. 
Varying A within this interval renders a smooth sample of SOO solutions. How¬ 
ever, for A S (0, A*) we can pose different SOOs whose solution lays exactly at 
the same place, as indicated by the gray fan in figure [^. If we measure anything 
about the SOO solutions, that quantity will be constant as a function of A for 
A € (0, A*) because we will persistently measure a property of the same design. 
That same property will vary smoothly over A € (A*, 1). At A* this quantity will 
be continuous but its derivative will not (figure]^, inset), as in second order 
phase transitions. In cases like this we say that the Pareto front ends abruptly 
at one of its extremes. Second order transitions also happen if the slope of the 
front spans d £ {d* > —oo,0) (i.e. if the opposite end of the front terminates 
abruptly) or if d G (—oo, d*_) U (d!j_, 0) (i.e. the front presents a sharp edge with 
an ill-defined derivative). 

A cavity in the front leads to first order phase transitions. At either side 
of the cavity in figure [J we find convex stretches whose points represent dif¬ 
ferent solutions for different SOO problems posed by different A G (0, A*) or 
A G (A*, 1). But right at A = A* (represented by the straight red line of figure 
Si) two solutions are SOO optima at the same time. This is a phase coexistence 
phenomenon characteristic of first order transitions. Pareto optimal solutions 
laying inside the cavity are bypassed and never get to be SOO optima. If we 
measure an oder parameter of the SOO solutions as a function of A (figure [^, 
inset), we find a gap resulting from the abrupt shift from one convex stretch of 
the front to the other at A = A*. 


2.1 Phase Transitions in Pareto Optimal Designs 

As examples, we choose two problems that have recently been treated from an 
optimization perspective. Take Complex Networks first, which are good models 
of a series of natural systems such as vascular or nervous circuits that 

might be constrained by physical costs (available material) while seeking the 
efficient implementation of biological function (e.g. distribution of nutrients). 
Some human made structures, such as transportation networks [3], would also 
benefit from optimal design. 
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Fig. 2. Pareto optimal networks with nodes spaced over a circle. A genetic 
algorithm was used to approximate the Pareto front (blue crosses and thick brown 
curve) of networks that minimize the average path length and the cost of their links, a 
The front implements a tradeoff between the clique and the minimum spanning tree. 
(Inset) The clique extreme of the front ends abruptly (see [18]) indicating a second 
order phase transition, b A cavity is revealed at the center of the front, which implies 
a first order phase transition, c Both transitions are revealed in the plot of any order 
parameter. (Second order at A 2 — 0.59, first oder at Aj ~ 0.34.) 


In [T3| we consider this problem to a greater extent. We take the cost ^( 7 ) 
of network 7 as a function of its edges (number or length) and its efficiency is 
accounted for by the average path length (1) ( 7 ), a naive proxy for how fast mes¬ 
sages can be relayed across the network. These are the targets for minimization 
{Tf = {ti = p{'y),t 2 = {1) ( 7 )}) that lead to a Pareto front and, depending on 
its shape, to phase transition and other interesting phenomena. In figure we 
represent the front for such an MOO along with some Pareto optimal networks. 
In this example the nodes are spaced over a circle and the cost of each link is 
proportional to its Euclidean distance. This front ends abruptly (figure]^, inset) 
and a cavity is present (figure [^, see m for discussion). This implies, corre¬ 
spondingly, a second and a first order transitions at A 2 — 0.59 and at A* ~ 0.34. 
These transitions can be noted in the plot of any order parameter (figure [^). 

Our second example explores the evolutionary constraints of human language, 
an unsettled challenge for the scientific community. The optimization of linguistic 
structures brings together universal language properties (such as Zipf’s law) and 
the presence of ambiguity, likely as a compromise between language economy and 
a large ability to talk about the outer world . Such a tension was proposed by 
Zipf himself [Bj and its mathematical formalization |5] leads to an MOO problem 
that was always treated as an SOO. Accordingly, phase transitions were readily 
identified but some debate lasted concerning its nature and meaning [27] . In | 8 |, 
languages a G A are modeled through a set of signals S and objects R whose 
associations are encoded in a matrix a = {aij} with = 1 if signal Si G S names 
object rj G R and 0 otherwise. This binary matrix presents many ones in a row 
if a signal is polysemous and many ones in a column if several words name the 
same object - i.e. if there are any synonymous. Every object is recalled equally 
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often and, if an object has many names, a speaker chooses uniformly among 
these when necessary. Two quantities are relevant (see one entropy hnia) 

associated to the uncertainty of a message when a hearer has to decode it - i.e. 
what object it is meant after the speaker has uttered a signal; and ii) another 
entropy hs{a) associated to the speaker choosing the right word to name an 
object among those available. A speaker might be allowed to be vague (as in 
“it” referring to any object) or she might be requested to be specific (perhaps 
finding the precise technicism in a scientific context). 



Fig. 3. Least effort languages, a Arbitrary Pareto optimal languages (blue crosses) 
lay on the straight line t 2 = 1 — ti. A straight front is a sign of criticality along a first 
order phase transition scenario. Either phase represents respectively the best scenario 
for the speaker (A < A'^, where communication is impossible unless through the context) 
and for the hearer (A > A'^, with high memory demands). Only at the critical point is 
a wide complexity available. Any order parameter (b, mutual information between the 
signals and the external world; c, effective vocabulary size) reflects the phase transition. 


These two entropies represent the effort made by hearers or speakers when 
using a language. They act as minimization targets (Ty = {ti = h//(a),t 2 = 
hs{a)}), so that languages a G A are subjected to a MOO. A subset (a^ € 
n) C A of object-signal associations implements the Pareto front, the optimal 
tradeoff between the efforts of a hearer and a speaker. This front is a straight 
line in target space (figure Hh)- Attending to the theory sketched above this 
is akin to a first order phase transition (see [H]) with one end of the front 
being the global optimum for A < A° and the other extreme of the front being 
optimum for A > A°. Right at A = A'^, a sudden jump happens between these 
two, very distinct phases. This can be appreciated in any order parameter as 
a sharp discontinuity (figure]^, c). The extremes correspond to i) a language 
that minimizes the speaker’s efforts for A < A'” (one single signal names every 
object, as in the “it” example before, so that the speaker does not need to think 
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the right association every time) and ii) a language that minimizes the hearer’s 
effort for A > A*^, with perfect pairings between signals and objects so that there 
is not any ambiguity when decoding the messages. 

Communication is difficult in both extremes, either because the signals con¬ 
vey little information about the objects (A < A'’, figure [3^), or because of the 
memory needs to browse a vast vocabulary (A > A*^, figure]^). Besides, we know 
that more complicated structures exist in real languages. These structures can be 
found right at A = A'”. A straight Pareto front is an indication of criticality ( |18j. 
[21]). In such cases, exactly at the critical value A'^, the whole front is also SOO 
optimal. Note that in usual first order transitions those solutions laying at the 
cavity are skipped altogether, while here a plethora of them becomes available. 
In [27] the authors proved that the global SOO minimizers at A = A*” consist of 
all possible languages without synonyms, hence these must constitute the Pareto 
front. More importantly, among these possible languages it exists one such that 
the frequency of the signals obeys Zipf’s law, as in natural human languages. 


3 Discussion 

3.1 Thermodynamics as an MOO-SOO Problem 

Thermodynamics is one of the best established branches of physics and dates 
back to more than two centuries ago. In its modern form - as statistical mechan¬ 
ics - it allows us to make precise predictions about diverse macroscopic physical 
phenomena. Its applications extend beyond physics, as complex systems are in¬ 
creasingly being investigated through maximum entropy models [21] , [30] . In [IB] 
we rewrite thermodynamics as an MOO-SOO problem, not to suggest that our 
theoretical framework modifies it in any way. Rather the opposite: By checking 
that our framework reproduces a robust physical theory, we strand our findings 
in a more solid ground. 

Phase transitions in complex systems often raise heated debate: being strict, 
phase transitions are defined for thermodynamics alone, through partition func¬ 
tions, and involve fluctuations that compel us to take a thermodynamic limit. 
Little can be done against such epistemological stand. This is yet another reason 
why we undertake the task of writing thermodynamics as an MOO-SOO. Such a 
formalization of statistical mechanics reproduces all the results concerning phase 
transitions in the exact same way that transitions arise in other MOO-SOO sce¬ 
narios. We suggest and support that the phase transition phenomenology arising 
in other MOO-SOO systems is more than a qualitative similarity. 

The argument is not repeated here because of space constraints, but the 
idea is to show that the independent, simultaneous minimization of internal en¬ 
ergy and maximization of entropy leads to a Pareto front subjected to the phe¬ 
nomenology found in section [^ In thermodynamics we deal with given physical 
systems that cannot be modified. We test probabilistic descriptions that tell us 
how likely it is to find the system in each part of its phase space. We wonder 
which of these descriptions present a lower internal energy and larger entropy. 
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Thus our design space X is the set of all possible probabilistic descriptions of 
the system under research. From that optimization we obtain a Pareto front 
whose shape (through cavities) and differential geometry (through sharp edges) 
imply phase transitions if the targets (ti = U, t 2 = S) were collapsed into an 
SOO problem. But that is precisely what happens in equilibrium thermodynam¬ 
ics through the minimization of the free energy F = U — TS [2Dj. We identify 
f2 = F, Xi = 1, and A 2 = —T, and the theory exposed above applies with 
transitions at singular temperature values. 

We insist that the optimization operates upon probabilistic descriptions of 
the thermodynamic species - while the shape of the front is determined by the 
properties of the physical system. It might be interesting to segregate what 
thermodynamic phenomenology happens because thermodynamic systems are 
probabilistic ensembles (in this regard they are unlike Pareto optimal networks 
or least effort languages, as much as networks and languages are unlike each 
other) and what phenomenology arises because of the shape of a Pareto front 
(that would yield the same phenomenology irrespective of the kind of designs 
considered - were they networks or languages - as long as the front had the same 
shape). 

This interpretation of statistic-mechanical systems is illustrated with two 
very simple examples with first and second order transitions and one critical 
point in |18| . As stated above, this is not to prove new thermodynamic results, 
but to provide more solid basis for this theory regarding MOO-SOO situations. 
Indeed, the role of cavities in first order phase transitions dates back to Gibbs 
[19], [20], whose Gibbs surface represented the states of a thermodynamic species. 
That surface is associated to the microcanonical ensemble [T5| and may be con¬ 
cave or convex. Its convex hull is associated to the canonical ensemble (hence 
to equilibrium at given temperature through free energy minimization), which 
is always convex. At cavities in the Gibbs surface, the description of both en¬ 
sembles must differ (as noted by the theory of ensemble inequivalence |3T|) and 
first order transitions occur. 

3.2 Closing Remarks 

With our recent findings [TH] we close a gap between the MOO literature, re¬ 
search on SOO tradeoffs, and statistical mechanics. On the one hand, standard 
MOO analysis does not take into account phenomena like phase transitions or 
criticality which, we believe, add up to our knowledge and enrich the description 
of Pareto optimal designs. On the other hand, analyses of the Pareto front are of¬ 
ten qualitative or based on subjective appreciations of its shape. The formalism 
developed in section [^ allows us to locate quantitatively very relevant details 
of the systems under research. These features shall persist under transforma¬ 
tions of the targets and, if not, the qualitative description would tell us how do 
these phenomena disappear. Furthermore, a solid connection to thermodynamics 
has been established. We are pretty confident of the immutable, lasting nature 
of thermodynamics; thus we can guess that, through the Pareto formalism, we 
have located broad features that unite the description of diverse MOO problems. 
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A prominent field for MOO application is biology IS], 0,0. Thermodynamic- 
like phenomenology is not discussed in these references, but the stage looks great: 
Is there a place for true MOO in biology? Against this, natural selection concerns 
itself with fitness maximization alone. This feels like an exciting MOO-SOO pic¬ 
ture, but we cannot guarantee linear global functions as in equation Beyond 
linearity, new phenomenology might be uncovered. 

Finally, an important, though conceptually difficult issue was left aside in m 
and only incidentally dealt with here. How do critical systems look like under 
an MOO perspective? Can we recover the astounding phenomena of criticality? 
This will be tackled in future work [^. Other theoretical aspects of MOO remain 
open to research. 
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